In the nineteenth century, the cheap seats at the back of a theatre or in the upper balcony.
The upper balcony in racially segregated venues such as a theatre to which black patrons were restricted.
Any source of heckling, unwelcome commentary or criticism, especially from a know-it-all or of an inexpert nature. May also now refer to general audience response: "Let's hear it from the peanut gallery."
Enough already from the peanut gallery; if you think you can do a better job, go right ahead.